[Feature] Added performance testing tool based on the PyTest testing framework #295
Conversation
test/test_uc_performance
Outdated
| case_hit_rate_map — {case_idx: hit_rate} 的映射 | ||
| """ | ||
| print(f"[INFO] 共计 {len(test_cases)} 个测试用例待执行") | ||
| failed_case = [] |
There was a problem hiding this comment.
failed_case is not used in this function.
|
put this file in benchmark dir seems better |
|
Pursuant to the UCM code-repository guidelines, all code comments must be composed in English. |
There was a problem hiding this comment.
You need to add the pip packages you use along with their versions to make it easier for others who don't have them (e.g., pandas, pydantic) to use them.
| server_url: "http://141.111.32.70:9382" | ||
| tokenizer_path: "/home/models/QwQ-32B" | ||
| # Performance Test Configuration | ||
| llmperf_test_cases: |
There was a problem hiding this comment.
- Configuration items can be added, referring to the implementation of logs and reports, with results stored using timestamps. (They can be uniformly placed in the reports directory to prevent too many subdirectories, and there should be an llmperf flag.)
- Parameter names such as max_num_completed_requests and num_concurrent_requests are not descriptive enough; additional descriptions should be added.
| from common.llmperf.utils.utils import reset_prefill_cache | ||
|
|
||
|
|
||
| def run_test_cases(test_cases, timestamp_dir, model, server_url, tokenizer_path): |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Menglths
changed the title
Menglths
force-pushed
the
develop
branch
2 times, most recently
from
55e817f to
f01918a
Compare
Wwwzff
left a comment
Wwwzff
left a comment
There was a problem hiding this comment.
we may pull the latest code from main branch and rebase your commit to the top
Menglths
force-pushed
the
develop
branch
2 times, most recently
from
3bd5375 to
ebf4639
Compare
All commits have been rebased. |
Menglths
force-pushed
the
develop
branch
2 times, most recently
from
21a07b1 to
8a90e76
Compare
5 participants
Purpose
Performance testing tool based on the PyTest testing framework
Modifications
1、Added tests for UC-related performance metrics, including full throughput and incremental throughput.
2、Support for custom PC hit rate.
3、Support for custom tokenizer.
Test